CLAIMS 

The invention claimed is: 



1 . A method of image processin g comprising the steps of: 

(a) determining a probability that an input pixel is one of a 
background and a foreground pixel of said image by 
comparing said input pixel to a background reference pixel 
model; and 

(b) revising said probability for said input pixel according to a 
probability that a neighboring pixel is a pixel of one of said 
background and said foreground and a relationship of said 
input pixel and a pixel structure included in said image. 



2. The method of claim 1 wherein the step of determining a probability that an 
input pixel is one of a background and a foreground pixel of said image by 
comparing said input pixel to a model of a background reference pixel 
comprises the steps of: 

(a) comparing a difference between said input pixel and said 
background reference pixel model to a threshold difference; 

(b) determining said probability that said input pixel is one of said 
background and said foreground pixels from a first probability 
function if said difference at least equals said threshold 
difference; and 

(c) determining said probability that said input pixel is one of said 
background and said foreground pixels from a second 
probability function if said difference does not at least equal 
said threshold difference. 
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3. The method of claim 2 wherein said difference between said input pixel and 
said background reference pixel model comprises a difference in a chromatic 
value of said input pixel and said background reference pixel model. 

5 4. The method of claim 2 wherein said threshold comprises a chromatic value. 

5. The method of claim 1 wherein the step of revising said probability for said 
input pixel according to a probability that a neighboring pixel is a pixel of one 
of said background and said foreground and a relationship of said input pixel 
and a pixel structure included in said image comprises the steps of: 

(a) filtering said image to identify a pixel structure included in 
said image; 

(b) assigning a first probability that a pixel neighboring said 
input pixel is a pixel of one of said background and said 
foreground as said revised probability for said input pixel, 
if said input pixel is included in said structure; and 

(c) if said input pixel is not included in said structure, 
assigning a second probability that a pixel neighboring 
said input pixel is a pixel of one of said background and 
said foreground as said revised probability for said input 
pixel. 

6. The method of claim 1 wherein said background reference pixel model is a 
mean of a pixel of a background reference image and at least one input pixel 

25 of an image background. 

7. The method of claim 1 further comprising the step of revising said background 
reference pixel model if said input pixel is a pixel of said background. 
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8. 



The method of claim 1 further comprising the step of substituting a new pixel 
for said input pixel in said image if said revised probability determines that 
said input pixel is more probably a pixel of said background. 



* 9. A method of segmenting an image, said method comprising the steps of: 

(a) capturing said image comprising a plurality of input pixels 

representing a background and a foreground of a scene; 

(b) capturing a background reference image comprising a plurality 
of background reference pixels representing said background 
of said scene; 

(c) classifying an input pixel as one a pixel of said background and 
said foreground according to a relationship of said input pixel 
to a background reference pixel; 

(d) identifying a structure in said input image; and 

(e) revising said classification of said input pixel according to a 
membership of said input pixel in said structure. 

10. The method claim 9 wherein the step of classifying an input pixel as one of a 
background and a foreground pixel according to a relationship of said input 
pixel to a background reference pixel comprises the steps of: 

(a) determining a first probability that said input pixel is a pixel of 
one of said background and said foreground if a relationship of 
said input pixel and said background reference pixel at least 
equals a threshold; and 

(b) determining a second probability that said input pixel is a pixel 
of one of said background and said foreground if said 
relationship of said input pixel and said background reference 
pixel does not at least equal to said threshold. 
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11. The method of claim 10 wherein at least one of said first and said second 
probabilities is a function of a difference between said input pixel and said 
background reference pixel. 

12. The method of claim 1 1 wherein said difference between said input pixel and 
said background reference pixel comprises a difference between a chromatic 
value of said input pixel and a chromatic value of said background reference 
pixel. 

1 3. The method of claim 12 wherein at least one of said chromatic values of said 
input pixel and said background reference pixel comprises a sum of a plurality 
of chromatic components of said pixel. 

14. The method of claim 10 wherein said relationship of said input pixel and said 
background reference pixel comprises a difference between a chromatic value 
of said input pixel and a chromatic value of said background reference pixel. 

15. The method of claim 14 wherein at least of one said chromatic values of said 
input pixel and said background reference pixel comprises a sum of a plurality 
of chromatic components of said pixel. 

16. The method of claim 14 wherein said chromatic value of said background 
reference pixel comprises a mean of a plurality of chromatic values for a 
plurality of background pixels included in a plurality of images. 

17. The method of claim 10 wherein said threshold comprises a function of a 
chromatic value for said background reference pixel. 
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18. The method of claim 17 wherein said function of said chromatic value of said 
background reference pixel comprises a function of a standard deviation of a 
chromatic value of a plurality of background pixels included in a plurality of 
images. 

5 

19. The method of claim 9 wherein the step of identifying a structure in said 
image comprises the step of filtering said image with a morphological filter to 
identify a plurality of connected input pixels in said image. 

The method of claim 9 wherein the step of revising said classification of said 
input pixel according to a membership of said input pixel in said structure 
comprises the steps of: 

(a) revising one of said first and said second probabilities 
according to a first relationship if said input pixel is a pixel 
included in said structure; and 

(b) revising one of said first and said second probabilities 
according to a second relationship if said input pixel is not a 
pixel included in said structure. 

The method of claim 20 wherein at least one of said first and said second 
relationships comprises a probability that a pixel neighboring said input pixel 
is a pixel of one of said background and said foreground. 

22. The method of claim 20 wherein said first relationship comprises a minimum 
25 probability that one of a plurality of pixels neighboring said input pixel is a 
background pixel. 
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23. The method of claim 20 wherein said second relationship comprises a 
maximum probability that one of a plurality of pixels neighboring said input 
pixel is a background pixel. 

24. The method of claim 9 further comprising the step of revising a background 
reference pixel according to a relationship of said background reference pixel 
and an input pixel. 

25. The method of claim 24 wherein the step of revising said background 
reference pixel according to a relationship of said background reference pixel 
and an input pixel comprises the step of determining a mean of a chromatic 
value of said background reference pixel and a chromatic value of said input 
pixel if said input pixel is classified as a pixel of said background. 

26. A method of segmenting an image comprising the steps of: 

(a) capturing said image comprising a plurality of input pixels 
representing a background and a foreground of a scene; 

(b) capturing a background reference image comprising a plurality 
of background reference pixels representing a background of 
said scene; 

(c) determining a probability that an input pixel is a pixel of one of 

said background and said foreground according to a 
relationship of a value measuring said input pixel and a value 
measuring a background reference pixel; 

(d) identifying a pixel structure in said image; 

(f) revising said probability for said input pixel according to a 
membership of said input pixel in said pixel structure and a 
probability that a neighboring pixel is a pixel of one of said 
background and said foreground; and 
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(g) revising said value measuring said a background reference 
pixel according to a relationship of said background reference 
pixel and said input pixel. 

27. The method claim 26 wherein the step of determining a probability that an 
input pixel is a pixel of one said background and said foreground according to 
a relationship of a value measuring said input pixel and a value measuring a 
background reference pixel comprises the steps of: 

(a) determining a first probability that said input pixel is a pixel of 
one of said background and said foreground if a relationship of 
said values measuring said input pixel and said background 
reference pixel at least equals a threshold; and 

(b) determining a second probability that said input pixel is a pixel 
of one of said background and said foreground if said 
relationship of said values measuring said input pixel and said 
background reference pixel does not at least equal to said 
threshold. 

28. The method of claim 27 wherein at least one of said first and said second 
probabilities is a function of a difference between said values measuring said 
input pixel and said background reference pixel. 

29. The method of claim 28 wherein said difference between said values 
measuring said input pixel and said background reference pixel comprises a 
difference between a chromatic value of said input pixel and a chromatic 
value of said background reference pixel. 
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30. The method of claim 29 wherein at least of one said chromatic values of said 
input pixel and said background reference pixel comprises a sum of a plurality 
of chromatic components for said pixel. 

5 31. The method of claim 27 wherein said relationship of said values measuring 
said input pixel and said background reference pixel comprises a difference 
between a chromatic value of said input pixel and a chromatic value of said 
background reference pixel. 

The method of claim 31 wherein at least one said chromatic values of said 
input pixel and said background reference pixel comprises a sum of a plurality 
of chromatic components for said pixel. 

The method of claim 31 wherein said chromatic value of said background 
reference pixel comprises a mean of a chromatic value for a plurality of 
background pixels included in a plurality of images. 

The method of claim 27 wherein said threshold comprises a relationship for a 
chromatic value for said background reference pixel. 

The method of claim 24 wherein said relationship for said chromatic value of 
said background reference pixel comprises a function of a standard deviation 
of a chromatic value for a plurality of background pixels included in a plurality 
of images. 

The method of claim 26 wherein the step of identifying a pixel structure in said 
image comprises the step of filtering said image with a morphological filter to 
identify a plurality of connected input pixels in said image. 
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37. The method of claim 26 wherein the step of revising said probability for said 
input pixel according to a membership of said input pixel in said pixel structure 
and a probability that a neighboring pixel is a pixel of one of said background 
and said foreground comprises the steps of: 
5 (a) assigning a first probability value to said input pixel if said input 

pixel is a pixel included in said structure; and 
(b) assigning a second probability value to said input pixel if said 
input pixel is not a pixel included in said structure. 

1 0 38. The method of claim 37 wherein at least one of said first and said second 

probability values comprises a probability that one of a plurality of neighboring 
pixels is a pixel of one of said foreground and said background. 

~-JLS 

W 39. The method of claim 37 wherein at least one of said first and said second 
m 15 probability values comprises one of a minimum and a maximum probability 
i that one of a plurality of neighboring pixels a pixel of one of said foreground 

=H and said background. 
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40. The method of claim 26 wherein the step of revising said value measuring 



yj ' 

^ 20 said a background reference pixel according to a relationship of said 
M= background reference pixel and said input pixel comprises the step of 

determining a mean of said value measuring said background reference pixel 
and said value measuring said input pixel if said input pixel is a pixel of said 
background. 



41 . The method of claim 40 wherein at least one of said values measuring said 
input pixel and said background reference pixel comprises a chromatic value 
for said pixel. 
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42. The method of claim 41 wherein at least one said chromatic value of said 
input pixel and said background reference pixel comprises a sum of a plurality 
of chromatic components for said pixel. 

43. A method of processing an image comprising the steps of: 

(a) capturing an image comprising a plurality of input pixels 
representing a background and a foreground of a scene; 

(b) capturing a background reference image comprising a plurality 
of background reference pixels representing a background of 
said scene; 

(c) determining a probability that an input pixel is a pixel of one of 
said background and said foreground according to a 
relationship of said input pixel and a background reference 
pixel; 

(d) identifying a pixel structure in said image; 

(f) revising said probability for said input pixel according to a 
membership of said input pixel in said pixel structure and a 
probability that a neighboring pixel is a pixel of one of said 
background and said foreground; and 

(g) substituting a new background pixel for a spatially 
corresponding input pixel classified as a background pixel. 

44. The method claim 43 wherein the step of determining a probability that an 
input pixel is a pixel of one of said background and said foreground according 
to a relationship of said input pixel and a background reference pixel 
comprises the steps of: 

(a) determining a first probability that said input pixel is a pixel of 
one of said background and said foreground if a relationship of 
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said input pixel and said background reference pixel at least 
equals a threshold; and 
(b) determining a second probability that said input pixel is a pixel 
of one of said background and said foreground if said 
relationship of said input pixel and said background reference 
pixel does not at least equal to said threshold. 

45. The method of claim 44 wherein at least one of said first and said second 
probabilities is a function of a difference between said input pixel and said 
background reference pixel. 

46. The method of claim 45 wherein said difference between said input pixel and 
said background reference pixel comprises a difference between a chromatic 
value of said input pixel and a chromatic value of said background reference 
pixel. 

47. The method of claim 46 wherein at least one of said functions of said 
chromatic values of said input pixel and said background reference pixel 
comprises a sum of a plurality of chromatic components for said pixel. 

48. The method of claim 44 wherein said relationship of said input pixel and said 
background reference pixel comprises a difference between a chromatic value 
of said input pixel and a chromatic value of said background reference pixel. 

49. The method of claim 48 wherein at least one said chromatic values of said 
input pixel and said background reference pixel comprises a sum of a plurality 
of chromatic components for said pixel. 



-23- 



50. The method of claim 48 wherein said chromatic value of said background 
reference pixel comprises a mean of a plurality of chromatic values for a 
plurality of background pixels included in a plurality of images. 

51 . The method of claim 44 wherein said threshold comprises a function of a 
chromatic value for said background reference pixel. 

52. The method of claim 51 wherein said function of said chromatic value of said 
background reference pixel comprises a function of a standard deviation of a 
chromatic value for a plurality of background pixels included in a plurality of 
images. 

53. The method of claim 43 wherein the step of identifying a structure in said 
image comprises the step of filtering said image with a morphological filter to 
identify a plurality of connected input pixels in said image. 

54. The method of claim 43 wherein the step of revising said probability for said 
input pixel according to a membership of said input pixel in said pixel structure 
and a probability that a neighboring pixel is a pixel of one of said background 
and said foreground comprises the steps of: 

(a) revising one of said first and said second probabilities 
according to a first relationship if said input pixel is a pixel 
included in said structure; and 

(b) revising one of said first and said second probabilities 
according to a second relationship if said input pixel is not a 
pixel included in said structure. 
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55. The method of claim 54 wherein at least one of said first and said second 
relationships comprises a probability that a pixel neighboring said input pixel 
is a pixel of one of said background and said foreground. 

5 56. The method of claim 54 wherein at least one of said first and said second 

relationship comprises one of a maximum and a minimum of a probability that 
a pixel neighboring said input pixel is a pixel of one of said foreground and 
said background. 

10 57. The method of claim 43 further comprising the step of revising a background 
reference pixel according to a relationship of said background reference pixel 
and an input pixel. 



58. The method of claim 57 wherein the step of revising said background 
15 reference pixel according to a relationship of said background reference pixel 
and an input pixel comprises the step of determining a mean of a chromatic 
Ut value of said background reference pixel and a chromatic value of said input 

□ pixel if said input pixel is classified as a pixel of said background. 



M 20 vl59. A method of replacing an image background in a video sequence comprising 
the steps of: 

(a) capturing an image comprising a plurality of input pixels 
representing a background and a foreground of a scene; 

(b) capturing a background reference image comprising a plurality 
25 of background reference pixels representing a background of 

said scene; 

(c) classifying an input pixel as a pixel of said background if a 
difference between said input pixel and a background 
reference pixel is less than a threshold difference; 
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(d) filtering said image to identify a pixel structure included in said 
image; 

(e) revising said classification of said input pixel according to a 
first probability relationship if said input pixel is a pixel of said 
pixel structure; 

(f) revising said classification of said input pixel according to a 
second probability relationship if said input pixel is not a pixel 
of said pixel structure; 

(g) revising a background reference pixel if said revised 
classification of a spatially corresponding input pixel classifies 
said input pixel as a background pixel; 

(h) substituting a new background pixel for said input pixel if said 
input pixel is classified as a pixel of said background; and 

(i) capturing another image comprising a plurality of input pixels 

representing a background and a foreground of said scene. 

60. The method of claim 59 wherein the step of classifying an input pixel as a 
pixel of said background if a difference between said input pixel and a 
background reference pixel is less than a threshold difference comprises the 
steps of: 

(a) determining a difference between a chromatic value for said 
input pixel and a chromatic value for said background 
reference pixel; and 

(b) comparing said difference to a threshold chromatic value. 

61. The method of claim 60 wherein at least one of said chromatic values of said 
input pixel, said background reference pixel and said threshold comprises a 
sum of a plurality of chromatic components. 
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62. The method of claim 60 wherein said threshold is a function of a standard 
deviation of a chromatic value of said background reference pixel. 

63. The method of claim 60 wherein said chromatic value of said background 
5 reference pixel comprises a mean of a chromatic value of a plurality of 

background pixels from a plurality images. 

64. The method of claim 59 wherein the step of revising said classification of said 
input pixel according to a first probability relationship if said input pixel is a 

10 pixel of said pixel structure comprises the step of assigning to said input pixel 
a probability that one of a plurality of neighboring pixels is a background pixel 
if said input pixel is a pixel of said structure. 

65. The method of claim 59 wherein the step of revising said classification of said 
input pixel according to a second probability relationship if said input pixel is a 
pixel of said pixel structure comprises the step of assigning to said input pixel 
a probability that one of a plurality of neighboring pixels is a background pixel 
if said input pixel is a pixel of said structure. 

66. The method of claim 59 wherein the step of revising a background reference 
pixel if said revised classification of a spatially corresponding input pixel 
classifies said input pixel as a background pixel comprises the step of 
calculating the mean of a chromatic value of said background reference pixel 
and said spatially corresponding input pixel of said background. 
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